Interior point methods and simulated annealing for nonsymmetric conic optimization

This thesis explores four methods for convex optimization. The first two are an interior point method and a simulated annealing algorithm that share a theoretical foundation. This connection is due to the interior point method’s use of the so-called entropic barrier, whose derivatives can be approximated through sampling. Here, the sampling will be carried out with a technique known as hit-and-run. By carefully analyzing the properties of hit-and-run sampling, it is shown that both the interior point method and the simulated annealing algorithm can solve a convex optimization problem in the membership oracle setting. The number of oracle calls made by these methods is bounded by a polynomial in the input size. The third method is an analytic center cutting plane method that shows promising performance for copositive optimization. It outperforms the first two methods by a significant margin on the problem of separating a matrix from the completely positive cone. The final method is based on Mosek’s algorithm for nonsymmetric conic optimization. With their scaling matrix, search direction, and neighborhood, we define a method that converges to a near-optimal solution in polynomial time

An Algorithm for Nonsymmetric Conic Optimization Inspired by MOSEK

We analyze the scaling matrix, search direction, and neighborhood used in MOSEK's algorithm for nonsymmetric conic optimization [Dahl and Andersen, 2019]. It is proven that these can be used to compute a near-optimal solution to the homogeneous self-dual model in polynomial time.Comment: 29 page

Complexity analysis of a sampling-based interior point method for convex optimization

We develop a short-step interior point method to optimize a linear function over a convex body assuming that one only knows a membership oracle for this body. The approach is based on Abernethy and Hazan's sketch of a universal interior point method using the so-called entropic barrier [arXiv 1507.02528v2, 2015]. It is well-known that the gradient and Hessian of the entropic barrier can be approximated by sampling from Boltzmann-Gibbs distributions, and the entropic barrier was shown to be self-concordant by Bubeck and Eldan [arXiv 1412.1587v3, 2015]. The analysis of our algorithm uses properties of the entropic barrier, mixing times for hit-and-run random walks by Lovász and Vempala [Foundations of Computer Science, 2006], approximation quality guarantees for the mean and covariance of a log-concave distribution, and results from De Klerk, Glineur and Taylor on inexact Newton-type methods [arXiv 1709.0519, 2017]

An Analytic Center Cutting Plane Method to Determine Complete Positivity of a Matrix

We propose an analytic center cutting plane method to determine if a matrix is completely positive, and return a cut that separates it from the completely positive cone if not. This was stated as an open (computational) problem by Berman, D\"ur, and Shaked-Monderer [Electronic Journal of Linear Algebra, 2015]. Our method optimizes over the intersection of a ball and the copositive cone, where membership is determined by solving a mixed-integer linear program suggested by Xia, Vera, and Zuluaga [INFORMS Journal on Computing, 2018]. Thus, our algorithm can, more generally, be used to solve any copositive optimization problem, provided one knows the radius of a ball containing an optimal solution. Numerical experiments show that the number of oracle calls (matrix copositivity checks) for our implementation scales well with the matrix size, growing roughly like $O(d^2)$ for $d\times d$ matrices. The method is implemented in Julia, and available at https://github.com/rileybadenbroek/CopositiveAnalyticCenter.jl.Comment: 16 pages, 1 figur

Simulated annealing with hit-and-run for convex optimization: rigorous complexity analysis and practical perspectives for copositive programming

We give a rigorous complexity analysis of the simulated annealing algorithm by Kalai and Vempala [Math of OR 31.2 (2006): 253-266] using the type of temperature update suggested by Abernethy and Hazan [arXiv 1507.02528v2, 2015]. The algorithm only assumes a membership oracle of the feasible set, and we prove that it returns a solution in polynomial time which is near-optimal with high probability. Moreover, we propose a number of modifications to improve the practical performance of this method, and present some numerical results for test problems from copositive programming

An analytic center cutting plane method to determine complete positivity of a matrix

We propose an analytic center cutting plane method to determine whether a matrix is completely positive and return a cut that separates it from the completely positive cone if not. This was stated as an open (computational) problem by Berman et al. [Berman A, Dur M, Shaked-Monderer N (2015) Open problems in the theory of completely positive and copositive matrices. Electronic 1. Linear Algebra 29(1):46-58]. Our method optimizes over the intersection of a ball and the copositive cone, where membership is determined by solving a mixed-integer linear program suggested by Xia et al. [Xia W, Vera JC, Zuluaga LF (2020) Globally solving nonconvex quadratic programs via linear integer programming techniques. INFORMS J. Comput 32(1):40-561 Thus, our algorithm can, more generally, be used to solve any copositive optimization problem, provided one knows the radius of a ball containing an optimal solution. Numerical experiments show that the number of oracle calls (matrix copositivity checks) for our implementation scales well with the matrix size, growing roughly like O(d(2)) for d x d matrices. The method is implemented in Julia and available at https://github.com/rileybadenbroek/CopositiveAnalyticCenter.jl. Summary of Contribution: Completely positive matrices play an important role in operations research. They allow many NP-hard problems to be formulated as optimization problems over a proper cone, which enables them to benefit from the duality theory of convex programming. We propose an analytic center cutting plane method to determine whether a matrix is completely positive by solving an optimization problem over the copositive cone. In fact, we can use our method to solve any copositive optimization problem, provided we know the radius of a ball containing an optimal solution. We emphasize numerical performance and stability in developing this method. A software implementation in Julia is provided

Interior point methods and simulated annealing for nonsymmetric conic optimization

This thesis explores four methods for convex optimization. The first two are an interior point method and a simulated annealing algorithm that share a theoretical foundation. This connection is due to the interior point method’s use of the so-called entropic barrier, whose derivatives can be approximated through sampling. Here, the sampling will be carried out with a technique known as hit-and-run. By carefully analyzing the properties of hit-and-run sampling, it is shown that both the interior point method and the simulated annealing algorithm can solve a convex optimization problem in the membership oracle setting. The number of oracle calls made by these methods is bounded by a polynomial in the input size. The third method is an analytic center cutting plane method that shows promising performance for copositive optimization. It outperforms the first two methods by a significant margin on the problem of separating a matrix from the completely positive cone. The final method is based on Mosek’s algorithm for nonsymmetric conic optimization. With their scaling matrix, search direction, and neighborhood, we define a method that converges to a near-optimal solution in polynomial time